Efficient Implementation of Approximate Linear Programming

While the ALP may involve only a small number of variables, there is a potentially intractable number of constraints — one per state-action pair. As such, we cannot in general expect to solve the ALP exactly. The focus of this paper is on a tractable approximation to the ALP: the reduced linear program (RLP). Generation of an RLP relies on three objects: (1) a constraint sample size m, (2) a probability measure � over the set of state-action pairs, and (3) a bounding set N ∀ � . The probability measure � represents a distribution from which we will sample constraints. In particular, we consider a set X of m state-action pairs, each independently sampled according to �. The set N is a parameter that restricts the magnitude of the RLP solution. This set should be chosen such that it contains �r̃. The RLP is defined by